Effective Transductive Learning via PAC-Bayesian Model Selection

We study a transductive learning approach based on clustering. In this approach one constructs a diversity of unsupervised models of the unlabeled data using clustering algorithms. These models are then exploited to construct a number of hypotheses using the labeled data and the learner selects an hypothesis that minimizes a transductive PACBayesian error bound, which holds with high probability. Empirical examination of this approach, implemented with spectral clustering, on a suite of benchmark datasets, indicates that the new approach is effective and that on some datasets it significantly outperforms one of the best transductive learning algorithms known today.